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The annual Kazanjian Foundation's Awards Program for the 
Teaching of Economics (hereafter designated as the KFAP) for ele- 
mentary, secondary and college and university teachers was initi- 
ated in 1962,* During the first ten years of the program, approx- 
imately 400 cash prizes and honorable mention awards have been 
granted to elementary, secondary and college teachers throughout 
the United States. The KFAP has been recognized as having had a 
profound influence in increasing the economics content included in 
existing courses and programs in the schools of educators who 
received awards [!]• 

The specific purposes of the national KFAP are: (1) to stimu- 
late improvements in teaching economics, (2) to encourage 
teachers to develop descriptions of their economic education 
teaching experiences, (3) to provide recognition for outstanding 
economic education teaching practices, and (4) to foster a 
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widening and continuing exchange of reports of successful teach- 
ing experiences in economics among teachers at every level of edu- 
cation [2]. 

/although it is probably true that the vast majority of award- 
winning entries do in fact contribute to students' understanding of 
economics, nowhere in the economic education literature have 
these teaching materials been systematically examined, ^s 
regards to instructional objectives, most award-winning units give 
specific atLt;i*lion and homage to such statements as "the develop- 
m.ent of a good method of thinking about economic problems and 
the process of orderly problem solving, " or "the ability to evalu- 
ate and use both qualitative and quantitative evidence when con- 
flicting viewpoints and approaches are encountered. " Do the 
teaching materials in the KFi^P, in fact, help teachers carry out 
such objectives in the instructional process? 

Vvhile the purposes of the KFP^? are commendable, it has not 
been determined, as yet, whether the award-winning entries sig- 
nificantly contribute to students' understanding in economics. In 
addition, the extent to which these materials can be effectively 
adapted and modified for classroom use by teachers in a variety of 
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school settings and with different student population^, has not 
been established. 

THE PROBLEM 

The central problem of this study was to measure the effective- 
ness of selected elementary award-winning teaching materials in 
the KFJ^P on students' performance in economics. Specifically, 
three hypotheses were examined in the investigation: 

1. /^s measured by the Test of Elementary Economics (TEE), 
the mean change scores of the experimental groups who 
used the Kazanjian materials are not significantly differ<=^nt 
from those of the control group who did not use special 
materials. 

2. As measured by the TEE, the mean change scores of classes 
t3ught by teachers who received in-service instruction c^re 
not significantly different from those classes taught by 
teachers who did not receive in-service training. 

3. The correlation between the posttest scores of the teachers 
on the Test of Economic Understanding (TEU) and the 
oosttest scores of their students on the TEE, is not 



significantly different from zero. 

DESIGN OF THE STUDY 
In identifying appropriate procedures to analyze the central 
problem underlying the investigation, four basic steps were estab- 
lished and followed including (1) the selection of the control and 
experimental groups, (2) the selection of the Kazanjian teaching 
units, (3) the selection of the testing instruments, and (4) the 
statistical analysis of the data. Each of these steps Is explained 
in more detail below. 

Selection of Experimental and Control Groups 
Tha research design involved twelve elementary teachers in 
the Sauk Rapids, Minnesota, School System and their students in 
grades four through six, in experimental and control groups. Two 
experimental groups with a total of 171 students in eight classes 
and one control group of 92 students in four classes were selected 
for the sample population. In this study design, intact closses 
were randomly assigned to control and experimental groups. 

The eight experimental teachers and the four control teacher s 
wore similar in age, educational training/ teaching experience and 



school principal evaluation. Ml were considered good teachers, 
held bachelor's degrees, and had over seven years of teaching 
experience. Subsequent tests on selected student characteristics 
including socioeconomic level, age, and interest in social studies, 
showed no statistically significant differences between the students 
in the experimental and control classes.** ^11 experimental 
classes and teachers used the same Kazanjian instructional mate- 
rials. 

The control and experimental groups established and utilized 
in the study may be visualized as follows: 



EXPERIMENTAL 


CONTROL 


Cl 


C2 


C3 


Use of Kazanjian 


Materials used in Cj^ 


Continue with 


teaching materials 


plus in-service 


regular social 


without in-service 


training sessions 


studies program 


training 






Four Teachers 


Four Teachers 


Four Teachers 


and Classes 


and Classes 


and Classes 



An equal number of classes were included in the sample for each 
of the three groups under investigation. While students in both Cj 



and C2 treatment groups were to receive instruction based upon 
the same Kazanjian curriculum materials, only the four teachers in 
C2 participated in a daily, five-week in-service training session 
during the summer of 1972 preced.ng the experiment. Mditionally, 
the four teachers in C2 met together with the principal investigator 
of this study for approximatr/y two hours each month throughout 
the thirty-six weeks of the experiment to discuss strategies for 
effectively Implementing the Kazanjian materials in their class- 
rooms . 

Selection of Teaching Materials 
A random selection of four intermediate grade elementary teach- 
ing materials submitted to the KFAP was made from the total popula- 
tion of forty-nine winning entries from 1962-1971.*** Ail award- 
winning Kazanjian entries describing teaching experiences carried, 
out by intermediate grade elementary teachers in their classrooms 
between July 1 and June 30 of each school year are eligible for 
selection. The first step in the selection process was to identify 
all of the individual award-winning intermediate grade elementary 
teaching materials. The author's name for each entry was written 



on a card, and the cards were separated by school years. Each 
card was assigned a different identification number. ^ total of 
four entries were randomly drawn from the above school year clas- 
sifications, using a table of random numbers. The four Kazanjian 
teaching units selected were utilized in all experimental classes. 
The same sequence and time schedules for teaching each of the four 
instructional units were established and followed in all the experi- 
mental classes. 

Selection of Testing Instruments 

Teachers in both the control and experimental groups were pre- 
and posttested on the nationally normed and validated Test of Eco- 
nomic Understanding . (TEU), Form and B [4]. For purposes of 
this study, the TEU was judged to be an appropriate instrument to 
measure the teachers' performance in economics. 

The control and experimental group students in the twelve 
classes were pre- and posttested on the familiar Test of Elementary 
Eco nomics. (TEE) [5], The TEE contains forty multiple choice ques- 
tions with four options given for each item. The test was believed 
to have the most potential for evaluating elementary students' 
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performance in economics. 

In view of the necessity of using intact classes, the John 
Flanagan Test of General Ability (TOGA) , Form A, 4-6 [6] was ad- 
ministered to ail students in both control and experimental groups 
so that differences in ability levels could be determined. The 
TOGA yields both an IQ score and a grade expectancy score and is 
particularly suited as a test of general ability for elementary grade 
pupils. 

The experiment can be summarized as follows: 



GROUPS 


PRETEST 


TREATMENT 


POSTTEST 


Experimental (C^) 
Four Classes 


TOGA 
and 
TEEi 


Use of Kazanjian 
Teaching Materials 
Without In-Service 
Training (36 weeks) 


TEE2 


Experimental (C2) 
Four Classes 


TOGA 
and 
TEEi 


Use of Kazanjian 
Teaching Mlaterials 
With In-Service 
Training (36 weeks) 


TEE2 


Control (C3) 
Four Classes 


TOGA 
and 
TEEi 


Regular Social 
Studies Curriculum 
(36 weeks) 


TEE2 
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Statistical /Analysis of the Study Results 



-As Table 1 indicates, the students in the experimental classes 
were brighter, had more previous knowledge of economics and 
ended the instructional program with significantly greater knowl- 
edge in economics than did the students in the control group. 
Moreover, as shown in Table 2, when comparing the TEE perfor- 
mance of experimental students with the performance of control 
students by any one of two improvement measures, even greater 
significance in the difference was found. -A t-test on the absolute 
improvement scores (Post TEE - Pre TEE = ^ TEE) indicated signifi- 
cance below the .01 level.**** 



Means S.D. Means S.D. Means S.D. 

TOGA 53.51 12.34 54.67 12.67 51.37 11.42 2.07* 

Pre-TEE 12.85 4.17 13.60 4.10 11 .47 3.94 4.06** 

Post-TEE 18.35 6.59 21.04 5.93 13.37 4.55 10.76** 



Table 1 



Description of Test Results Between Groups of 
Elementary School Students 



Variables 



Pll Students Experimental Group Control Group 
(N=263) (N=171) (N=92) 



t-test 
Comparing 
Means 



••Significant at the .05 level 



** Significant at the .01 level 
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Table 2 



Post Test Score Improvement Measures 



Improvement 
Measure 



All 
Students 
(N=263) 



Experimental 
Students 
(N=171) 



Control 
Students 
(N-92) 



Absolute (1) 
Improvement 



5.50 



7.44 



1.90 



Percentage (2) 
Improvement 



42.8% 



54.6% 



16.5% 



(1) Post TEE * Pre TEE = A TEE 

(2) Pre TEE as the base 

Unfortunately/ this level of aggregation offers little explana- 
tion about the actual impact of the individual treatments on stu- 
dents' performance in economics. A more meaningful analysis of 
the data can be performed by making a comparison of the actual 
mean change TEE scores for classes in each treatment, adjusting 
these change scores to account for differences in mental ability. 
A one-way analysis of covariance test was employed for 12 classes 
treating eech class as an individual case or as the unit of sampling 
For each class , the mean on TOGA was treated as the covariate and 
i:he mean change score (TEEg - TEE^) was used as the criterion 
measure with the three treatments being materials plus in-sorvice, 
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materials, and control. The results of the test are presented In 
Table 3. 

Table 3 

Results of One-Way Analysis of Covariance for 

Elementary School Data 
U^I-=263) 

Source of Variation 



Between 



Within 



Total 



Sum of Squares- Y 


36.88 


366. 


76 


403 


.63 


Sum of Squares: X 


108.34 


33. 


61 


141 


.95 


Sum of Products 


62.96 


37. 


76 


25 


.20 


Degrees of Freedom 


2.00 


9. 


00 


11 


.00 


Mjusted Sum of Squares: X 


110.65 


29. 


72 


140 


.38 


Degrees of Freedom for 
adjusted Sum of Squares 


2.00 


8. 


00 


10 


.00 


Variance Estimates 


S^=55.33 


o2_ 


^3.72 







F = 55.33/3.72 = 14.89 p <.01 



The data reported in Table 3 indicated that the treatment effects 
were significant beyond the .01 level. This means that the amount 
of change and the effectiveness of the treatments in economics 
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achievement differed significantly between the experimental and 
the control groups. Quite clearly, almost all of the variation in 
the X means (mean change score on TEE) can be attributed to the 
influence of the treatments. 

It is of interest here to calculate directly the adjusted mean 
change scores on the TEE. These adjusted values can be inter- 
preted as estimates of the actual amount of change in economics 
performance that would have resulted if the treatment groups had 
equal TOGA scores. Table 4 presents these adjusted mean change 
va lues. 

Table 4 

Unweighted Group Means for Relevant Variables 

Mean* ' Mean* Unadjusted Mean* Adjusted 
Groups N TOGA Change Score Change Score 

(TEE2 - TEEj) (TEE2 - TEEi) 

Experimental (Cj) 85 53,39 5.78 5.77 

Experimental {C^ 86 55.71 9.17 9. 40 

Control (C3) 92 51.37 1 .82 2.03 

*is unweighted average of class means 

As shown in Table 4, these adjusted mean change scores on 
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TEE vary consideicbly , a fact that is clearly reflected in the large 
F-ratio obtained in the analysis of covariance test. These results 
confirm that the differences between the unadjusted mean change 
scores on TEE are due largely to the treatments. 

To obtain a more meaningful interpretation of the data reported 
in Table 4, a comparison of pairs of mean change scores on TEE, 
adjusted for TOGA, was made via t-tests. Such comparisons are 
used to identify pairs of means that may be significant from those 
that may not be. The results of the adjusted mean pair compari- 
sons among treatment groups are presented in Table 5. 



Table 5 



Results of Adjusted Mean Pair Comparisons 
for Treatment Groups 



Comparisons 



Degrees 
of 

Freedom 



T-value* 



Significance 




6 



2.35 



• 10 



Ci with C3 



8 



2.97 



.05 



Cj with C3 



6 



5.77 



.01 



-'/adjusted mean square used as the comon error term 



As shown in Table 5, all three adjusted mean pair comparisons 
achieved significance. The comparison that achieved the greatest 
significance was that between groups and C^. For this compari- 
son, the significance level was less than .01. It may be inferred 
from this data that both experimental treatments were significantly 
more effective than was the control method, and th^re is an indica- 
tion of superiority for the materials Plus in-service treatment . 

The data compiled and analyzed in the study were also fit into 
linear regression models. The equation for each of the models takes 

the familiar linear form wherein Y = a + ^i^i ^2^2 ^ ^3^3 

+ hy^j + e. Post TEE, the dependent variable, is assumed to be 
additively and linearly related to the following independent vari- 
ables:***** 

xi = Pre TEE (0 - 40), continuous 
X2 = Post TEU (0 - 50), continuous 
X3 = TOGA (0 - 85), continuous 

X4 = Teacher's Sex (0 = male, 1 = female) dichotomous 
Xg = Years of Teaching Experience (3 - 30), continuous 
xg = Educational Training (0-3, 0 = no degree), continuous 



X7 = Group Type 

A (0 = Experimental Ci and 1 - Control C3) dlchotomous 

B (0 = Experimental 1 = Experimental Cj^ and Control C3) 
dichotomous 

C (0 ~ Experimental C2» 1 = Experimental Cj^) dichotomous 

The independent variables used in the three regression models are 
listed and explained in Table 6. Model I, with post TEE as the 
dependent variable, is run against all of the measured independent 
variables which the students and teachers bring into the clossroom 
prior to instruction as weil as instructional outcome variables. Group 
types B and C (xy) are excluded from the independent variables in 
this regression equation. 

Model II stipulates post TEE as the dependent variable and includes 
the same independent variables as Model I with the exclusion of 
group type A, bcy) , and adds group type B (xy). 

Model III, with post TEE as the dependent variable, includes 
the independent variables of Models I and II with the exception of 
group types A and B (xy), and adds group type C (xy) . 

Regression Model I in Table 6 confirms that the use of the 
Kazanjian teaching materials does contribute significantly to the 
economic understandings of students — as measured by TEE for stu- 
dents in the study population. While controlling for prior knowl- 
edge in economics (xj.) with all the other independent variables, it 
was found that the residual contribution of the treatment (xy) had a 
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Table 6 



Regression Coefficients for Models I - HI 
O?" Ratio in Parentheses) 





Model I 


Model II 


Model III 




Dependent 
Variable 
Post TEE 


Dependent 
Variable 
Post TEE 


Dependent 
Variable 
Post TEE 


Independent Variables 


3f = 18.35 


3f = 18.35 


5r= 18.35 








xi Pre TEE 
(40 - 0) 
X = 12.85 


.33 
(22.68)** 


.32 
(20.28)** 


.40 
(20.33)** 


X2 Post TEU 
' (50 - 0) 
X =■■ 36.77 


.17 

(11.30)** 


.31 
(54.52)** 


.11 
(.81) 


X3 TOGA 
(85 - 0) 
X = 53.51 


.16 

(48.70)** 


.16 

(44.32)** 


.16 

(28.25)** 


x^ Teacher's Sex 
(1 = Female, 1-0) 
X = .56 


.13 
(1.01) 


.15 
(.98) 


-1.81 
(.87) 


xc Years of Experience 
(12 - 1) 
X = 9.60 


.21 
(.76) 


.24 
(.81) 


.23 
(.80) 


Xg £iClUCatlUil 

(1 = Bachelor's, 1-0) 
X = .92 


.13 
(.93) 


.24 

(.86) 


.15 
(.87) 


Xy Group 

A (0= Ci C2, 1 =03) 

X = .35 
B (0 = C2, 1 = Ci C3) 

X = .67 
C (0 = C2, 1 = Cj) 

X = . 50 


-7.68 
(48.32)** 


-4.24 
(40.81)** 


-2.50 
(20.37)** 


Constant 
Standard Error 
Adjusted R2 


5.72 
3.93 
.65 


2.30 
3.97 
.64 


6.92 
4.06 
.55 



** Significant at .01 level 
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.01 significance level for its F-value of 48. 32 ******* A regres- 
sion coefficient of -7.68 indicated that a student loses almost 
seven and three-quarters points on the post TEE as a result of 
taking the regular social studies program as compared with taking 
instruction based on the use of the Kazanjian teaching materials; 
the experimental groups were superior in their post TEE performance. 

It may be inferred from regression Models II and HI in Table 6 
that the students taught by teachers using the Kazanjian materials 
with in-service training, learned significantly more economics than 
did students taught by teachers without the in-service program. 
While holding constant prior knowledge in economics (xj), , mental 
ability (x^) , and any possible differential influence of the teacher 
(x2, X4, X5, X5), the results of regression Model II revealed that 
the residual impact of the in-service treatment (X7) was significant 
at the .01 level. regression coefficient of -4.24 indicated a lo5>s 
of approximately four and one-quarter points on the post TEE for a 
student taught by a teacher without in-service instruction. How- 
ever, as regression Model HI indicates, when comparing the value 
added on the post TEE for in-service experimental classes (C2) as 
against the performance on non-in-service experimental classes 
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(Ci), different significance arises. 

In Model III, a regression coefficient of -2. SO indicated a loss 
of only two and one-half points on the post TEE for a student in a 
non-in-service experimental class. The data suggest l:hat the dif- 
ferential impact of the in-service treatment on post TEE was rela- 
tively greater when comparing the performance of the in-service 
classes against that of the control classes, than against the non- 
in-service experimental classes. 

RELATIONSHIPS ^MONG TEACHER VARIi^BLES 
Table 7 gives insights into the influences of the independent 
teacher variables on post TEE. The results shown in this table 
revealed a significant correlation coefficient of .47 between the 
TEU2 and the TEE2. Regression Models I and II in Table 6 confirm 
the results that post TEU is significant in contributing to the stu- 
dents' economic understanding as measured by post TEE. Holding 
prior knowledge in economics constant with all other independent 
variables, it was found that the residual contribution of the teacher 
(post TEU) had a .01 significance level for its F-value in both 
Models I and II. In Model I, a regression coefficient of .17 
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indicated that for every incremental gain of one point on post TEU 
there is an approximate gain of .17 on post TEE, or in the students* 
economic understanding. Similarly, regression Model II revealed 
that an approximate gain of .31 on post TEE was achieved for every 
incremental gain of one point on post TEU. 

SUMMARY OF FINDINGS 
On the basis of the data compiled and analyzed In this investi- 
gation, the following findings were obtained: 

1. The mean change TEE scores of the experimental groups using 
the Kazan jian teaching materials were significantly different from 
those of the control group who did not use the materials, at the .01 
level. 

2. The mean change TEE scores for classes taught by teachers 
involved in in-service training were significantly different from those 
classes tought by teachers without in-service training, at the ,01 
level. 

3* The correlation between the teachers' post TEU scores and the 
students' post TEE scores was found to be significant at the .01 
level . 
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Table 7 

Correlations Among the Teacher Variables 

(N=263) 



TEE, TEUo TOGA Sex Years Education 



Posttest (TEE2) 


.54 .47 .46 


-.24 


.03 


.25 


Pretest (TEEj) 


.24 .49 


-.09 


.03 


-.11 


Posttest (TEU2) 


.06 


-.08 


.39 


-.30 


TOGA 




-.09 


.01 


.04 


Teacher's Sex 






.30 


-.22 



Years of Teaching Experience .62 
Teacher's Education 

CONCLUSIONS 

As always, one is well advised to resist generalizations based 
upon a single case study. Yet, some tentative implications emerge. 

The first and most obvious implication to be drawn from this 
study is that is needs to be replicated. In spite of this need and 
the tentativeness of the results, this study does have important 
implications both for the organization and teaching of elementary 
economics. This study has shown that the Kazanjian teaching 



materials contribute significantly to elementary students' under- 
standing of basic economic concepts and practices. Moreover, it 
appears c s if, when teachers use the Kazanjian materials in conjunc- 
tion with in-service training, the student picks up a comparative 
advantage in learning economics. However, inasmuch as the study 
revealed differential affects on students* performance between the 
two experimental groups and, to the extent that background on eco- 
nomics content is incorporated in the Kazanjian materials, the neces- 
sity for full-scale in-service economic education programs for ele- 
mentary teachers is reduced. 

The results of the study have shown that, while the pre TEE 
appears to be the best predictor of what a student will achieve on 
post TEE (correlation coefficient of .57), the teacher's competency 
in economics (as measured by post TEU) contributes significantly 
to the students* economic understanding. A correlation coefficient 
of .47 between TEE2 and TEU2 would indicate that there is a con- 
stant relationship between TEE and TEU--as one improves, so does 
the other. However, only 22 percent oi the variance in TEE can be 
explained by using TEU as an independent variable, and it is clear 
from Table 6 that other variables, some acting on both mec^,*^ures, 
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some only on one measure, contribute to explaining the variability 
of the two measures. But, the strength of the relationship between 
TEE and TEU indicates that if TEU shows improvement as a result of 
participation in an in-service course or from the utilization of 
Kazanjian teaching materials, the same participation or utilization 
will affect the TEE score. Suggestively, in-service teaching train- 
ing programs in economic education should continue to develop 
unique elementary curricular offerings and instructional materials 
designed to meet the needs of their heterogeneous clientele. 

Another major implication of this study is that the Kazanjian 
Foundation inwards Program should conMnue to encourage the devel- 
opment and use of innovative curricula? materials and instructional 
practices for the teaching of elementrry economics. The study 
results suggest that the award-winning elementary materials in the 
KFAP can be meaningfully adapted and effectively utilized by other 
classroom teachers in different school settings and with different 
student populations. Seemingly, the elementary Kazanjian materials 
have some transfar utility and should be used by teachers in struc- 
turing learning experiences suited to the particular needs of their 
students. 
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Finally, it is important to reiterate that there is a differential 
gap in student performance gains between the experimental and 
control groups, Although this study could not purport to explain 
this significant difference solely on the basis of its model or data, 
it does nevertheless provide some justification for these tentative 
implications. In the final analysis, this study may be of greatest 
value, not for the questions it answered, but for the direction it 
suggests for future inquiry. The impact of instruction and curri- 
cular materials on students' economic understanding is a crucial 
problem facing educational institutions of growing influence and 
should be dealt with in future research. 
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FOOTNOTES 

* Sponsored by the Calvin K. Kazanjian Foundation and administered 
by the loint Council on Economic Education, the KFAP is designed 
to stimulate improvements in the teaching of economics at all levels 
of instruction. A panel of seven judges composed of recognized 
leaders from economics and economic education established the 
criteria for the awards and selected the recipients. Cash prizes and 
honorable mention awards are made to teachers whose entries receive 
award recognition. The National Depository of Kazanjian Teaching 
Materials is housed in the Vernon R. Alden Library at Ohio Univer- 
sity, Athens, Ohio. 

**The U.S. Office of Education guidelines for determining a school's 
eligibility for financial aid under Title 1 of the Elementary and Sec- 
ondary Education Act (E.S^E.A,) are used as the criteria for classi- 
fying the socioeconomic level of the students in the participating 
schools. In addition, the Sauk Rapids School System utilizes a 
locally-developed instrument to determine student interest in all 
subject areas including social studies. See [3] for more complete 
statistical data and explanations of techniques of analysis used. 
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**'*For purposes of this study, winning entries in the KFAP included 
all cash prize and honorable mention awards. The winning entries 
for an eight year period from 1962-1963 to 1970-1971 are identified 
in the cumulative index of Economic Education Experiences of Enter - 
prising Teachers . Volume 9, published and distributed by the Joint 
Council on Economic Education, 

****Any discussion of gain added on the TEE must be qualified with 
the recognition that the gain-added function is clearly nonlinear. 
There are easy questions requiring knowledge of facts, questions 
of "medium" difficulty requiring comprehension of factual material, 
and some which are very difficult requiring application of knowledge 
to new situations. In fact, the test was specifically designed this 
way in terms of cognitive composition. It is therefore inappropriate 
to compare increments on this test as constructed. Only on a truly 
''linear test" can comparisons be safely made. 

*****/^ number of other possible independent variables were con- 
sidered for inclusion in thiG study. However, such other variables 
as the students' age, sex and family background v/ere found to be 



non-significant in similar studies [7]and/or inter- correlated with 
those independent variables which were identified in this study. 
Moreover, for purposes of generalizability, only those independent 
variables which were identified prior to the experiment were included 
in the analysis. 

****** The possibility of interaction between Education and Teach- 
ing Experience was also examined by including an interaction vari- 
able (xg • xg) in a subsequent regression. The results were essen- 
tially the same as found in Model I of Table 5. 
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